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The  purpose  of  this  study  was  to  identify  areas  of  instructional  methods,  techniques, 
strategies,  or  content  that  could  be  used  to  improve  the  program  effectiveness  of  the  job- 
oriented  basic  skills  (JOBS)  curricula.  Subjects  were  lower  aptitude  students  from  a 
variety  of  settings.  Four  different  study  agencies  administered  four  alternative 
strategies:  schema-based  studying,  learned  helplesness  alleviation,  instructional  sequenc¬ 
ing,  and  hierarchical  displays.  There  were  no  significant  results  from  any  one  study  and. 
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results  were  ambiguous  among  themselves.  It  is  concluded  that  the  four  studies 
merely  set  a  foundation  from  which  to  conduct  future  research  i*  the  area  of  basic 
skills  learning  by  lower  aptitude  students. 


FOREWORD 


This  research  was  conducted  under  contract  with  the  University  of  Illinois  at  Urbana- 
Champaign,  Syracuse  University,  and  University  of  Southern  California.  The  research  was 
in  support  of  work  unit  Z1176-PN.03  (Improved  Performance  Through  Instruction  in  "A" 
School  Related  Basic  Skills).  The  purpose  of  this  work  unit  is  to  determine  whether  a  job- 
oriented  basic  skills  (DOBS)  training  program  can  compensate  for  the  skill  deficiencies  of 
lower  aptitude  personnel  such  that  they  can  successfully  complete  Navy  technical  schools 
and  perform  to  standards  in  the  fleet. 

This  is  the  fifth  report  on  the  JOBS  training  program.  Previous  reports  (NPRDC  TRs 
81-24,  82-14,  83-5,  and  83-21)  described  program  development  and  evaluation.  This  report 
is  an  overview  of  four  research  studies  conducted  to  identify  instructional  methods, 
techniques,  strategies,  or  content  areas  that  could  be  used  to  improve  JOBS  program 
effectiveness. 

The  contracting  officer's  technical  representative  was  Meryl  S.  Baker. 


3.  W.  RENARD  JAMES  W.  TWEEDDALE 

Captain,  U.S.  Navy  Technical  Director 

Commanding  Officer 
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Problem 

Currently,  the  attrition  rate  of  job-oriented  basic  skills  (JOBS)  graduates  is 
approximately  twice  that  of  "A"-school-qualified  graduates  (21  vs.  10%).  Although  this 
differential  is  accounted  for  in  part  by  the  fact  that  JOBS  students  have  much  lower 
aptitude  scores  and  lower  post-"A"-school-attrition  rates  than  qualified  students,  methods 
to  increase  die  effectiveness  of  the  JOBS  curricula  are  continually  being  sought. 

Objective 

The  objective  of  this  effort  was  to  conduct  research  that  would  identify  instructional 
methods,  techniques,  strategies,  or  content  areas  that  could  be  used  to  improve  the  JOBS 
program  effectiveness. 

Approach 

Four  requests-for-proposal  were  funded  to  investigate  alternate  strategies  that  might 
be  used  to  enhance  the  JOBS  curricula. 

Results 


Results  among  die  four  studies  were  ambiguous. 
Conclusions 


The  four  studies  conducted  merely  set  a  foundation  from  which  to  conduct  future 
research  in  the  area  of  basic  skills  learning  by  low  aptitude  students.  It  is  clear  that  if 
significant  resources  were  to  be  invested  in  the  modification  of  the  JOBS  curricula,  they 
would  have  to  be  justified  by  a  far  broader  investigation  resulting  in  less  ambiguous 
results  than  the  four  studies  presented  in  this  report. 

Recommendations 

It  is  recommended  that: 

1.  Curriculum  development  personnel  not  alter  the  JOBS  curricula  at  the  present 
time. 

2.  Instructional  research  personnel  use  the  studies  presented  in  this  report  to 
delineate  future  areas  of  instructional  research. 


V 


CONTENTS 

Page 

INTRODUCTION  .  1 

Background  and  Problem  .  1 

Objective  . 1 

APPROACH  AND  RESULTS  .  1 


Study  1  .  1 

Purpose  . 1 

Hypothesis  .  2 

Procedure  and  Subjects . 2 

Findings .  2 

Study  2  .  3 

Purpose  and  Hypothesis  . 3 

Subjects . 3 

Procedure  .  4 

Findings .  4 

Limitations  of  the  Study .  5 

Study  3  .  6 

Purpose  and  Hypothesis  .  6 

Subjects  and  Procedures  .  7 

Methodological  Concerns  . 7 

Findings .  8 

Recommendations  .  9 

Study  4  .  9 

Purpose  .  9 

Subjects .  9 

Procedure  .  10 

Findings .  10 

Recommendations  .  11 

CONCLUSIONS  .  11 

RECOMMENDATIONS  .  II 

REFERENCES  .  13 


DISTRIBUTION  LIST 


15 


INTRODUCTION 


Background  and  Problem 

The  job-oriented  basic  skills  (JOBS)  program  was  conceived  in  1977  in  response  to 
predictions  that  there  would  be  a  shortage  of  high  quality  accessions  in  the  1980s.  High 
quality  accessions  are  those  individuals  who  score  in  mental  categories  I,  II,  or  upper  III  on 
the  Armed  Services  Vocational  Aptitude  Battery  (ASVAB)  tests.  The  purpose  of  the 
program  was  to  determine  whether  mastery  of  job-oriented,  basic/ prerequisite  skills 
training  would  provide  lower  aptitude  students  with  the  knowledge  they  needed  to  (1) 
complete  selected  "A"  schools  successfully  and  (2)  perform  to  fleet  standards  on  the  job. 
Prerequisite  skills  training  was  developed  for  four  "A"  schools:  Propulsion  Engineering 
(PE),  Operations  (OPS),  Administrative/Clerical  (A/C),  and  Electricity/Electronics  (E/E) 
(Harding,  Mogford,  Welching,  &  Showel,  1981). 

Currently,  the  attrition  rate  of  JOBS  graduates  is  approximately  twice  that  of  "A"- 
school-qualified  graduates  (21  vs.  10%)  (Baker  <3c  Hamovitch,  1983).  Although  this 
differential  is  accounted  for  in  part  by  the  fact  that  JOBS  students  have  much  lower 
aptitude  scores  and  lower  post-"A"-school-attrition  rates  than  do  qualified  students, 
methods  to  increase  the  effectiveness  of  the  JOBS  curricula  are  continually  being  sought. 

A  recent  evaluation  of  the  JOBS  program  (Baker  &  Hamovitch,  1983)  has  shown  that 
curricula  improvements  could  potentially  attenuate  Navy  technical  shortages  and  contrib¬ 
ute  to  minority  upward  mobility.  Further,  although  aptitude  differences  may  preclude 
JOBS  students  from  ever  attaining  the  "AN-school-qualified-student  level,  JOBS  curricula 
improvements  may  enhance  their  subsequent  "A"  school  and  fleet  performance. 

Objective 

The  objective  of  this  effort  was  to  conduct  research  that  would  identify  instructional 
methods,  techniques,  strategies,  or  content  areas  that  could  be  used  to  improve  the  JOBS 
program  effectiveness. 


APPROACH  AND  RESULTS 

Four  requests-for-proposal  were  funded  to  investigate  alternate  strategies  that  might 
be  used  to  enhance  the  JOBS  curricula.  A  brief  overview  of  each  of  these  efforts,  as 
adapted  from  the  contractors'  original  reports,  is  presented  below. 

Study  1  (Anderson,  jt  Armbruster.  1982) 

Purpose 

The  purpose  of  this  study  was  to  teach  JOBS  trainees  (1)  an  appropriate  higher  order 
organizing  scheme  they  could  use  to  learn,  remember,  and  use  Navy  technical  manuals  and 
(2)  a  systematic  study  strategy  they  could  use  in  conjunction  with  the  higher  order, 
organizing  scheme. 

Hypotheses 

This  study  is  based  on  schema  research  that  hypothesizes  that  (1)  students  can 
improve  their  learning  from  text  after  being  taught  basic  text  organizational  patterns  and 
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(2)  study  techniques  are  effective  only  if  students  use  the  schema  techniques  as  an  aid 
when  processing  important  information  in  a  meaningful,  memorable  form.  The  promising 
results  of  research  in  which  students  improved  their  learning  from  text  after  being  taught 
basic  text  organizational  patterns  provided  part  of  the  impetus  for  the  present  investiga¬ 
tion.  It  was  thought  that  JOBS  trainees  would  benefit  from  learning  the  organizational 
patterns  of  technical  materials.  In  this  study,  the  term  "frame*1  refers  to  a  higher  order, 
general  organizational  scheme  for  a  particular  type  of  content.  A  frame  has  slots  for  the 
subcategories  of  content  associated  with  it.  A  frame  with  its  constituent  slots  can  be 
instantiated  by  the  information  in  a  particular  text  that  deals  with  the  type  of  content 
covered  by  the  frame. 

Procedure  and  Subjects 

The  first  task  identified  a  frame  for  the  type  of  technical  text  that  the  JOBS 
students  would  be  reading.  Propulsion  Engineering  "A"  school  materials  and  associated 
rate  training  manuals  were  selected  as  representative.  An  extensive  content  analysis  of 
these  materials  revealed  that  the  content  could  be  captured  by  a  frame  with  the  following 
slots:  (1)  name  of  the  object  (piece  of  equipment  or  part  of  a  system),  (2)  main  uses  of  the 
object,  (3)  parts  of  the  object  and  their  functions,  (4)  explanation  of  the  operation  of  the 
object  or  how  it  works,  and  (5)  problems  that  could  occur  with  the  object  and  how  to 
prevent  and/or  solve  them.  The  researchers  named  the  frame  OPPUN,  an  acronym 
standing  for  operation,  parts  and  functions,  problems  and  prevention/solution,  uses,  and 
name  (OPPUN). 


Based  on  a  technical  reading  pretest,  JOBS  students  who  served  as  subjects  were 
divided  into  two  groups:  experienced  students  (those  who  were  in  the  JOBS  curriculum 
program  for  3  or  more  weeks)  and  new  students  (those  who  started  the  JOBS  curriculum 
program  that  week).  Pairs  of  pretest  scores  were  used  to  randomly  rank  students  into 
experimental  and  control  groups.  Ranking  proceeded  from  the  highest  to  the  lowest  pairs 
of  scores,  first  from  the  experienced  groups  and  then  from  the  new-student  groups.  Four 
groups  of  students,  matched  for  ability  as  measured  by  the  pretest,  resulted:  (1)  control- 
experienced  students,  (2)  control-new  students,  (3)  experimental-experienced  students, 
and  (4)  experimental-new  students. 

Once  the  experimental  versus  control  groups  were  formed,  they  were  taught  different 
learning  strategies.  The  experimental  groups  were  introduced  to  the  OPPUN  concept  with 
which  they  used  frames  to  focus  on  important  information.  The  control  groups  were 
taught  a  Main-Idea  strategy  with  which  they  underlined  main  ideas  and  recorded  them  in 
the  form  of  notes.  Both  the  experimental  and  the  control  groups  were  given  ample  time 
over  a  period  of  days  to  practice  their  respective  study  strategies. 

Findings 

Results  indicated  that  the  OPPUN  study  strategy  seemed  to  facilitate  the  new 
students  in  the  parts  section  but  was  detrimental  in  the  operations  section.  In  contrast, 
the  OPPUN  strategy  facilitated  die  experienced  students  in  the  operation  section  but  was 
slightly  detrimental  in  the  parts  section.  In  addition,  the  experienced  students  who  used 
the  OPPUN  strategy  scored  substantially  higher  than  their  Main-Idea-control  group  or 
their  new-student-comparison  group.  The  OPPUN  group  scored  24  percent  higher  than  the 
Main-Idea  group  on  the  uses  content  and  162  percent  higher  on  the  problem-solution 
content.  In  a  sense,  the  pattern  reflected  by  the  experimental  students  supported  the 
hypothesis  that  the  OPPUN  strategy,  when  compared  with  the  Main-Idea  strategy,  would 
result  in  more  facilitative  learning  outcomes. 


An  unusual  attrition  pattern  caused  interpretation  problems.  Even  at  the  beginning 
of  the  experiment,  there  were  not  very  many  students  available.  The  attrition  problem 
was  most  pronounced  with  die  experienced  students  in  the  OPPUN  strategy  group. 
Initially,  this  group  was  the  largest  with  11  students.  However,  by  the  third  training 
session,  there  were  only  four  students— the  same  four  who  took  the  final  exam.  This 
represents  a  64  percent  attrition  rate  for  this  group,  whereas  none  of  the  other  three 
groups  experienced  nearly  as  large  a  loss. 

The  attrition  problem  suggests  an  alternative  explanation  of  the  results,  which 
showed  that  the  OPPUN  group  performed  better  on  the  posttest  than  did  the  Main-Idea 
group.  That  is,  the  strategy  treatment  may  have  had  nothing  to  do  with  the  difference; 
rather,  those  students  who  dropped  out  of  the  experiment  were  perhaps  the  less  intelligent 
ones  who  needed  the  night  study  sessions  to  do  remedial  work.  The  four  students  who 
remained  in  the  study  were  perhaps  brighter  than  those  who  remained  in  the  Main-Idea 
group.  To  test  this  hypotheses,  pretest  scores  of  the  students  in  these  two  groups  were 
compared.  A  comparison  of  means  showed  a  difference  in  the  hypothesized  direction 
(27 .2  compared  with  24.2),  but  the  difference  was  not  significant,  t(12)  =  1.079,  p  >  .30. 
Therefore,  the  hypothesis  that  the  groups  who  actually  took  the  posttests  would  differ  in 
reading  ability  was  rejected. 

The  data  suggest  that  the  OPPUN  strategy  worked  better  with  the  JOBS  students  who 
had  been  in  the  JOBS  curriculum  for  at  least  3  weeks  (experienced  students)  than  it  did 
with  the  new  students.  This  may  be  a  result  of  the  JOBS  training  already  received  by  the 
experienced  group.  Perhaps  the  exposure  to  other  types  of  learning  strategies  during  the 
JOBS  course  enhanced  their  ability  to  employ  the  OPPUN  strategy. 

Certainly  the  findings  are  too  tentative  to  suggest  that  the  OPPUN  strategy  be 
incorporated  into  the  JOBS  curriculum.  More  research  at  several  levels  is  needed  before 
such  a  decision  can  be  made. 


Study  2  (Keller,  1982) 

Purpose  and  Hypothesis 

The  purpose  of  this  study  was  to  test  an  instructional  design  strategy  that  would  help 
eliminate  the  interference  effect  of  "learned  helplessness"  on  the  acquisition  of  basic 
skills  by  low  aptitude  students.  Learned  helplessness  is  a  phenomenon  created  by  a 
perceived  independence  between  response  and  reinforcement.  It  develops  when  an 
organism  is  repeatedly  subjected  to  situations  in  which  it  is  motivated  to  succeed,  but  it  is 
in  fact  helpless  to  do  so.  Once  the  perception  of  helplessness  is  established,  it  persists 
into  situations  in  which  success  is  possible.  The  development  of  a  learned  helplessness 
attitude  results  in  low  expectancy  for  success.  This  becomes  a  self-fulfilling  prophecy  for 
students  who  continue  to  fail  despite  having  had,  at  times,  a  real  desire  to  succeed. 

Subjects 

The  trainees  in  the  JOBS  program  would  seem  to  have  "learned  helplessness" 
characteristics.  They  were  low  aptitude  persons  who  typically  had  a  sense  of  failure  in 
regard  to  school-related  subjects,  such  as  math  and  reading.  Furthermore,  it  was  difficult 
to  distinguish  between  a  lack  of  native  ability  versus  years  of  low  achievement  as  causes 
of  low  measured  ability.  To  test  the  assumption  of  a  learned  helplessness  factor  among 
JOBS  trainees,  subjects  were  subjected  to  an  instructional  strategy  designed  to  alleviate 


learned  helplessness.  It  was  conjectured  that,  if  such  a  treatment  were  successful,  it 
would  support  the  inference  that  a  learned  helplessness  condition  was  present  among  these 
learners. 


JOBS  students  were  unavailable  for  participation  in  this  study.  Subjects  were  instead 
drawn  from  a  group  of  recruits  who  scored  below  the  6.4  reading  grade  level  on  the  Gates- 
MacGinitie  Reading  Comprehension  Test  administered  by  the  Navy  and,  hence,  had  been 
referred  to  the  academic  remedial  training  (ART)  program. 

Procedure 


Subjects  were  administered  the  reading  interest  survey  (Keller  Sc  Keller,  1981),  used 
to  measure  values,  expectancies,  and  attributions,  and  Level  II  of  the  wide  range 
achievement  test  (WRAT)  (Jastak,  3.  S.  Sc  Jasak,  F.,  1978),  used  to  assess  recognition  and 
naming  of  letters  and  pronunciation  of  words  out  of  context. 

Subjects  were  divided  into  two  groups:  reattribution  treatment  and  success  only. 
The  reattribution  treatment  group  completed  four  instructional  segments  in  reading,  were 
administered  WRATs  following  each  segment,  and  were  given  both  positive  and  reattribu- 
tional  feedback.  Hence,  the  students  in  this  group  were  not  only  praised  for  their 
performance  but  they  were  also  counseled  that  any  good  performance  was  a  direct  result 
of  their  efforts  and  not  of  some  external  environmental  factor.  The  success-only  group 
received  only  positive  feedback  after  they  successfully  completed  an  instructional 
segment  in  reading  and  during  the  WRAT  readministration. 

Upon  completion  of  all  reading  instruction  and  the  fourth  admnistration  of  the 
WRAT,  both  groups  were  administered  the  Peabody  Individual  Achievement  Test  (PIAT) 
(Dunn  &  Markwardt,  1970),  which  included  vocabulary  and  comprehension  subtests,  and 
the  Krodman  scale  of  self-esteem,1  and  were  readministered  the  reading  interest  survey.2 

Findings 

Results  offered  additional  confirmation  of  a  basic  alleviation  strategy  for  learned 
helpless  attitudes  among  low  aptitude,  or  at  least  measured  low  aptitude,  learners.  The 
study  a’  offered  support  for  a  general  expectancy-value  theory  explanation  of  the 
motivational  effects  of  the  strategy,  and  it  appeared  to  illustrate  the  effects  of  the 
strategy  on  personal  attributions  as  predicted  by  a  reformulated  theory  (Abramson, 
Seligman,  Sc  Teasdale,  1978).  The  subjects  in  the  reattribution  group  showed  a  greater 
shift  toward  internal  attributions  and  they  definitely  improved  their  performance.  By 
inference,  the  effectiveness  of  this  strategy  seemed  to  help  confirm  the  presence  of  a 
learned  helplessness  attitude  among  learners  who  had  been  consistent  failures. 


‘Keller,  3.  M.,  Sc  Keller,  B.  H.  Development  of  parallel  self-report  and  teacher 
report  measures  of  student  motivation  for  specific  school  subjects.  New  York:  Syracuse 
University,  Instructional  Design,  Development,  and  Evaluation  Program.  Occasional 
paper,  1981. 

2Krodman,  D.  A 
attritions  associated 
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i.  Negative  effect,  self-esteem  deficits,  and  the  internality  of 
with  helplessness  (Unpublished  doctoral  dissertation).  New  York: 
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Learned  helplessness  appeared  to  be  a  difficult  condition  to  reverse,  particularly 
when  it  had  been  generalized  to  many  different  facets  of  a  particular  skill  area 
(Seligman,  1976).  The  present  study  had  a  positive  influence  within  a  short  time,  but  it 
was  impossible  to  know  whether  this  influence  generalized  beyond  the  experimental 
setting.  Even  in  this  short  time  span,  several  important  conditions  were  observed.  First, 
it  was  not  easy  to  provide  an  instructional  unit  of  work  at  the  appropriate  level  of 
challenge,  especially  in  reading.  The  ART  program  helped  make  this  possible  because  of 
the  detailed  pretesting  that  was  conducted  and  the  instructional  program  that  was 
coordinated  with  the  skills  covered  on  the  pretest.  In  a  typical  classroom,  it  might  be 
even  more  difficult  to  provide  an  instructional  unit  of  work  at  the  appropriate  level  of 
challenge  because  of  a  greater  degree  of  heterogeneity  among  the  students  and  an 
inability  to  manage  the  number  of  subgroups  that  would  result. 

Another  factor  that  influenced  the  practical  significance  of  results  was  the  testing 
and  feedback.  A  student  must  be  tested  soon  after  honest  effort  has  been  exerted,  and 
the  feedback  must  follow  immediately  after  the  testing  if  the  results  are  to  be 
significant.  This  is  another  situation  that  is  difficult  for  the  typical  instructor  to  manage. 
Although  instructors  could  offer  supportive,  concerned  feedback,  they  could  not  do  so  on  a 
precise  schedule  because  of  multiple  demands  on  their  attention  during  any  given  class 
period.  Military  instructors,  particularly,  tend  to  have  rather  large  classes  for  short 
periods  of  time,  especially  in  "academic"  subjects.  Alleviation  would  be  easier  to 
implement  in  small  classes,  such  as  the  skill  development  classes  in  which  an  instructor 
works  closely  with  a  small  group  of  trainees. 

Limitations  of  the  Study 

There  were  several  limiting  factors  in  this  study.  First,  the  general  environment  in 
ART  appeared  to  be  so  supportive  from  a  motivational  standpoint  that  it  may  have 
interfered  slightly  with  the  effects  of  the  treatment.  The  nature  of  the  interference  was, 
presumably,  positive  as  it  may  have  elevated  the  posttest  measures  of  both  groups.  This 
is  desirable  from  an  ethical  point  of  view,  but  it  may  have  contributed  to  a  type  of 
"ceiling"  effect  here. 

A  second  problem  concerned  the  revolving  nature  of  the  procedure.  The  subjects 
from  both  the  reattribution  and  the  success-only  groups  freely  mingled  in  their  classrooms 
at  lunch  and  after  classes.  This  could  have  introduced  a  biasing  effect,  even  though  the 
informal  observation  of  the  experimenters  did  not  reveal  such  biasing.  The  experimenters 
talked  with  the  subjects  and  the  instructors  at  the  end  of  their  participation  in  the  study. 
In  no  case  were  these  persons  aware  of,  or  even  particularly  inquisitive  about,  the  critical 
features  of  the  study.  They  wanted  to  know  the  general  purpose,  how  it  was  being 
conducted,  and  what  its  effect  would  be  on  the  recruits,  but  none  of  this  seemed  to  have 
led  to  a  specific  source  of  confounding.  It  could,  however,  have  led  to  a  general 
Hawthorne  effect.  The  presence  of  the  investigators  and  the  nature  of  the  study  may 
have  contributed  to  a  heightened  sense  of  arousal,  and  that  may  have  accounted  for  some 
of  the  changes  in  both  groups,  but  it  would  not  seem  to  have  had  a  biasing  effect  on  the 
differential  treatment  results.  Even  so,  it  would  be  beneficial  to  conduct  the  study  under 
conditions  in  which  there  were  no  reactive  effects  due  to  the  presence  of  an  unfamiliar 
"experimenter." 

One  other  limiting  factor  that  should  be  mentioned  is  that  the  study  conducted 
entirely  with  males  limits  its  generalizability.  There  is  no  particular  reason  to  believe 
that  the  strategy  would  work  differently  or  less  well  with  females,  as  a  prior  research 
study  found  no  sex  differences  in  this  area.  However,  this  will  have  to  be  considered  as  a 
limitation  until  there  is  a  broader  base  of  support  for  the  strategy. 


Study  3  (Rcigeluth,  1982) 

Purpose  and  Hypothesis 

The  purpose  of  this  investigation  was  to  examine  the  effect  on  performance  of 
various  sequence  and  synthesis  approaches  to  basic  skill  instruction.  Four  sequence 
(forward  chaining,  backward  chaining,  hierarchical,  and  elaboration)  and  two  synthesis 
(performance  synthesis  and  contextual  synthesis)  techniques  were  investigated,  definitions 
for  which  are  given  below.  Sequence  and  synthesis  techniques  were  hypothesized  to  be  of 
varying  effectiveness,  depending  on  length  of  task  to  be  learned.  Hence,  four  tasks  of 
varying  lengths  were  investigated.  Tasks  were  in  the  math  and  English  subject  areas. 

1.  Sequence  Techniques. 

a.  Forward  chaining.  This  form  takes  the  results  of  an  information  processing 
task  analysis  and  sequences  the  steps  in  the  order  in  which  they  are  performed.  That  is, 
the  step  that  is  performed  first  is  the  step  that  is  taught  first.  The  rationale  for  this 
sequence  is  the  logic  of  the  correspondence  between  the  performance  sequence  and  the 
learning  sequence. 

b.  Backward  chaining.  This  form  (Gilbert,  1962)  takes  the  results  of  an 
information  processing  analysis  and  sequences  the  steps  in  the  opposite  order  from  that  in 
which  they  were  performed.  That  is,  the  step  that  is  performed  last  is  the  step  that  is 
taught  first.  All  the  necessary  inputs  from  prior  steps  (which  have  not  yet  been  taught) 
are  provided  so  that  the  student  can  actually  practice  performing  that  step.  The  rationale 
for  this  sequence  is  that  the  completion  of  the  task  is  far  more  intrinsically  rewarding  to 
the  student  than  the  completion  of  some  intermediate  step.  A  backward  chaining 
sequence  is  designed  to  result  in  greater  reinforcement,  which  in  turn  should  improve 
learning.  It  also  seems  likely  that  a  constant  awareness  of  the  goal  or  purpose  or  outcome 
of  what  is  being  learned  provides  more  meaningfulness  and  motivation. 

c.  Hierarchical.  This  form  takes  the  results  of  a  hierarchical  task  analysis  and 
sequences  them  in  a  bottom-up  order.  Ordinarily,  the  sequence  starts  with  the  left-most 
skill  on  the  bottom  of  the  hierarchy  and  proceeds  to  the  right  until  all  of  the  skills  that 
are  subordinate  to  a  single,  higher  level  skill  have  been  taught.  The  rational  for  the 
hierarchical  sequence  is  that  lower  level  skills  are  components  of  the  higher  level  skills 
and  therefore  must  be  learned  before  the  higher  level  skills  can  be  learned. 

d.  Elaboration.  This  form  is  based  on  the  results  of  both  an  information 
processing  analysis  and  another  form  of  analysis  called  path  analysis.  This  kind  of 
sequence  is  only  used  for  a  rule  that  has  at  least  one  decision  step  and  hence  at  least  two 
H^r  more  branches.  Path  analysis  entails  the  identification  of  all  possible  "paths"  (or 
•distinct  combinations  of  steps)  that  could  be  used  to  perform  the  rule.  Then  the  shortest 

path  is  taught  first,  usually  in  forward  chaining  sequence.  Also,  each  step  in  that  path  is 
always  preceded  by  any  prerequisites  that  were  revealed  by  a  hierarchical  analysis  of  it. 
Then  the  next  shortest  path  is  taught  in  the  same  manner,  and  so  on,  until  all  of  the  paths 
have  been  taught.  The  rationale  for  this  sequence  is  that  the  learner  is  actually  able  to 
perform  the  whole  task  (albeit  under  the  simplest  possible  circumstances)  from  the  very 
first  lesson. 

2.  Synthesis  Techniques. 

a.  Contextual.  This  form  shows  how  the  various  pieces  of  content  fit  together. 
In  the  case  of  rules,  it  shows  the  order  among  the  steps  or  operations.  Contextual 
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synthesis  is  usually  presented  before  the  related  application-level  instruction,  although  it 
may  come  after.  At  the  very  beginning  of  the  instruction,  it  may  show  the  relationships 
between  the  rule  that  is  about  to  be  taught  and  similar  rules  that  have  already  been 
taught,  in  which  case  it  is  usually  called  "external”  synthesizer.  Otherwise,  it  shows 
where  the  step  (or  set  of  steps)  that  is  about  to  be  taught  fits  within  the  whole  rule  that  is 
being  taught.  In  this  case,  it  is  usually  called  an  "internal"  synthesizer. 

b.  Performance.  This  form  is  intended  to  help  the  student  to  perform  the 
whole  rule  smoothly,  without  hesitation  and  with  full  recall  of  which  step  comes  when.  It 
is  not  enough  for  students  to  simply  learn  all  the  individual  steps;  it  is  essential  for  them 
to  know  when  to  use  and  when  not  to  use  each  step  in  relation  to  all  the  other  steps. 
Performance  is  a  habit  that  comes  through  practice.  Hence,  performance  synthesis 
always  comes  after  the  application-level  instruction,  and  it  always  includes  "integrated" 
generalities,  examples,  and  practice.  Integrated  generalities  describe  the  order  in  which 
to  use  all  the  steps  taught  so  far  in  the  instruction,  including  when  to  use  and  not  to  use 
each.  Integrated  examples  show  the  use  of  all  those  steps  to  solve  a  specific  problem,  and 
it  shows  them  all  being  used  back-to-back.  Similarly,  an  integrated  practice  item  requires 
the  use  of  all  those  steps  to  solve  a  specific  problem,  and  it  requires  that  they  be  used 
back-to-back. 

Subjects  and  Procedure 

As  JOBS  students  were  unavailable,  college  freshmen  enrolled  in  a  remedial  algebra 
course  or  in  a  basic  English  course  served  as  subjects.  Subjects  were  primarily  remedial 
learners  rather  than  first-time  learners.  In  other  words,  the  purpose  of  the  study  was  to 
review  skills  that  the  learners  had  already  been  exposed  to,  rather  than  to  teach 
completely  new  skills.  This  difference  may  have  influenced  the  effects  of  both  contextual 
synthesis  and  sequence.  Since  the  student  needed  review  but  received  primary  instruc¬ 
tion,  the  results  of  this  study  may  be  entirely  different  from  what  they  would  have  been  if 
the  students  had  been  first-time  learners.  However,  for  purposes  of  generalization  to 
Navy  JOBS  students  who  are  remedial  learners,  this  problem  was  of  no  concern. 

Methodological  Concerns 


Subjects  lacked  familiarity  with  the  different  instructional  sequences  and  this  could 
have  had  a  negative  impact  on  both  achievement  and  effect  of  results.  Some  students 
initially  expressed  considerable  confusion  over  the  backward  chaining  sequence.  The  same 
may  also  have  been  a  factor  in  the  elaboration  sequence  and,  to  a  lesser  extent,  in  the 
hierarchical  sequence.  Thus,  lack  of  familiarity  likely  had  differential  effects  across  all 
treatments,  with  the  particular  detriment  of  all  sequences  except  forward  chaining. 

The  nature  of  the  math  test  given  and  the  experimental  situation  worked  to  reduce 
any  differences  that  might  otherwise  have  resulted  from  the  treatment  effects,  although 
the  math  test  was  that  normally  used  in  the  remedial  algebra  course  and  was  not  an  ideal 
measure.  First,  the  number  of  items  for  each  task  was  quite  small,  which  impeded  the 
test's  ability  to  discriminate  among  levels  of  achievement.  Second,  the  test  did  not 
measure  the  full  range  of  skills  that  had  been  taught.  Many  types  of  problems, 
particularly  the  more  difficult  ones  to  which  the  rules  taught,  were  not  included  on  the 
test  and  reduced  the  test's  ability  to  discriminate  among  different  levels  of  achievement. 
It  was  considered  unfortunate  that  the  experimental  environment  precluded  administra¬ 
tion  of  a  test  created  by  the  researcher. 

In  the  experimental  situation,  there  was  a  considerable  time  gap  between  the 
administration  of  the  treatments  and  the  test.  During  the  time  gap,  students  were 
allowed  to  take  their  booklets  home  and  study  them.  It  is  possible  that  they  compared 
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their  booklets  with  those  of  other  students.  Further,  and  perhaps  even  more  important,  a 
student  could  reread  his  booklet  in  any  sequence,  which  precluded  his  having  received  a 
true  implementation  of  his  instructional  sequence  because  he  had  already  read  steps  that 
appeared  later  in  the  booklet.  It  would  have  been  far  superior  to  have  administered  a  test 
immediately  after  the  students  had  first  studied  their  booklets  from  beginning  to  end. 

A  final  set  of  methodological  concerns  were  related  to  the  English  experiment.  The 
task  was  "how  to  write  an  argumentative  essay,"  and  the  achievement  measure  was  a 
student  essay.  All  essays  were  "blind*'  rated  on  a  scale  from  1  to  10  by  at  least  two 
experts— the  section  instructor  and  the  instructor  of  another  section  of  English  101. 
("Blind"  meant  that  none  of  the  raters  was  aware  of  the  ratings  of  any  of  the  other 
raters.)  The  two  ratings  differed  by  more  than  3  points  in  over  50  percent  of  the  essays. 
In  all  such  cases  (two  ratings  that  differed  by  more  than  3  points),  another  rater  provided 
a  third  blind  rating.  Even  though  the  only  criteria  used  in  the  ratings  were  those  that 
were  taught  by  the  treatment  materials,  the  ratings  were  poorly  correlated.  It  was  also 
found  that  instructors  rated  class  essays  consistently  higher  than  did  the  outside  rater. 
This  lack  of  an  objective  and  reliable  measure  of  achievement  made  it  very  difficult  to 
assess  the  reliability  of  differences. 

It  is  important  to  note  that  the  general  ability  level  of  Navy  JOBS  students  is  lower 
than  that  of  the  present  study  participants.  However,  it  should  also  be  noted  that  there 
were  no  interactions  between  ability  level  and  any  of  the  factors  investigated  in  this 
study.  Since  the  range  in  SAT  scores  in  our  sample  was  considerable  (300-710),  this  lack 
of  interaction  provides  some  support  that  findings  may  be  generalizable  to  the  Navy  JOBS 
students. 

Findings 

The  results  of  this  study  indicated  that  the  effects  of  both  sequence  and  synthesis 
techniques  varied  directly  with  the  length  of  the  task  to  be  learned,  with  absolutely  no 
effect  on  short  tasks.  Also,  the  forward  chaining  sequence  resulted  in  higher  performance 
achievement  than  did  the  elaboration  sequence.  No  other  significant  effects  were  found. 

Several  inter pretauons  may  be  suggested  by  the  results.  First,  it  seems  likely  that 
there  was  a  certain  minimum  quantity  of  interrelated  procedural  content,  above  which  an 
elaboration  approach  was  the  only  viable  way  to  go,  but  below  which  the  elaboration 
approach  did  not  function  as  well  as  alternatives.  Several  factors  may  have  influenced 
why  this  approach  was  dysfunctional  below  a  minimum  quantity  of  procedural  content. 
One  is  that  students  were  confused  when,  for  instance,  four  steps  were  taught  as  a  stand¬ 
alone  rule  in  the  first  lesson  and  other  intermediate  steps  were  taught  in  later  lessons, 
such  that  what  was  step  3  in  lesson  1  became  step  7  in  lesson  2,  and  so  forth.  This 
awkwardness  in  numbering  is  unavoidable  in  an  elaboration  approach,  unless  steps  are  not 
numbered  at  all. 


Another  factor  that  may  have  accounted  for  the  elaboration  approach  being  dysfunc¬ 
tional  below  a  minimum  quantity  of  procedural  content  was  the  nature  of  human  learning. 
Spacial  and  chronological  cues  may  be  powerful  aids  to  learning.  For  example,  when  a 
person  wishes  to  find  some  information  that  has  been  read  fairly  recently,  he  or  she  may 
remember  that  the  information  was  on  the  top  of  a  right-hand  page.  It  is  likely  that  such 
a  spacial  cue  was  encoded  in  what  Gagne  and  White  (1978)  referred  to  as  the  image 
memory  store.  In  a  similar  way,  a  sequence  of  events  (which  is  characteristic  of  any  role) 
is  believed  by  the  same  theorists  to  be  stored  in  the  episodic  memory  store.  These  are  in 
contrast  to  intellectual  skills  that  are  believed  to  be  stored  mainly  in  the  propositional 


memory  store.  It  seems  quite  possible  that  episodic  memory  may  have  a  limit  in  terms  of 
the  size  of  a  rule  that  it  can  comfortably  handle  at  a  given  time.  That  is,  if  the  memory 
limit  is  exceeded,  then  it  becomes  necessary  to  "internalize"  that  rule  with  it  being  passed 
to  the  propositional  memory  store,  which  is  organized  hierarchically  (or  subsumptively). 
The  episode  memory  then  benefits  from  a  subsumptive  or  elaborative  sequence  (i.e.,  a 
simple-to-complex  or  general-to-detailed  sequence).  If  this  interpretation  is  correct,  then 
an  important  area  for  future  research  would  be  to  determine  the  critical  size  below  which 
a  forward  or  backward  sequence  would  be  optimal  and  above  which  an  elaborative 
sequence  would  be  optimal. 

Recommendations 

Given  the  current  state  of  knowledge,  the  major  implications  cf  this  and  other 
related  research  for  the  design  of  instruction  around  intellectual  skills  in  general  and  on 
basic  skills  in  particular  are  as  follows. 

1.  Identify  (through  cut  information  processing  task  analysis)  all  rules  that  need  to 
be  taught.  Be  sure  to  combine  all  related  rules  into  a  single  large  rule,  with  the  result 
being  several  large  rules. 

2.  Decide  if  the  total  amount  of  instruction  required  to  teach  each  single  large  rule 
exceeds  approximately  20  hours.  If  it  does  not  exceed  20  hours,  then  use  forward  chaining 
(without  contextual  synthesis)  to  sequence  the  instruction. 

3.  If  the  total  amount  of  instruction  exceeds  20  hours,  then  use  an  elaboration 
sequence.  However,  the  elaboration  theory  currently  proposes  that  each  "lesson"  in  the 
elaboration  sequence  be  limited  to  about  1  hour's  worth  of  instruction.  This  research 
suggests  that  each  "lesson"  should  supply  at  least  10  hours,  or  more,  worth  of  instruction. 
This  would  probably  allow  for  an  entirely  independent  branch  of  a  rule  to  be  taught  in 
each  "lesson,"  such  that  the  problem  of  a  student  having  to  renumber  steps  would  not  be 
encountered. 

4.  Within  each  "lesson"  (10  hours  or  more  worth  of  instruction),  a  forward  chaining 
(with  contextual  synthesis)  or  a  backward  chaining  (without  contextual  synthesis)  sequence 
should  be  used. 

5.  Do  not  bother  to  use  either  contextual  or  performance  synthesis  within  any 
"lesson."  Rather,  use  them  at  the  beginning  and  end  of  each  "lesson." 

Study  »  (Allan.  A  Merrill,  1982) 

Purpose 

The  purpose  of  this  study  was  to  explore,  on  a  theoretical  level,  the  use  of  several 
aptitude  approaches  to  predict  the  degree  of  external  support  required  by  learners.  On  an 
empirical  level,  the  study  attempted  to  demonstrate  the  feasibility  of  moderate  aptitude 
students  being  guided  in  the  selection  of  learning  strategies  appropriate  to  a  specific 
learning  task. 

Subjects 


Because  JOBS  students  were  unavailable,  subjects  were  selected  from  an  area  other 
than  the  targeted  population.  Subjects  were  from  a  continuation  high  school  in  Fullerton, 
California.  The  degree  to  which  the  skill  and  knowledge  level  of  these  students  reflected 
that  of  the  JOBS  student  population  was  not  discussed  by  the  investigators. 


Procedure 


The  study  focused  on  a  direct  comparison  of  system-assigned  and  student-assigned 
strategies,  with  attempts  to  control  for  differences  in  aptitude.  System-assigned 
strategies  are  those  that  are  delivered  over  a  separate,  independent  variable  information 
channel  and  that  exclude  specific  reference  to  subject-matter  content.  Three  strategies 
implemented  in  this  study  were  those  for  (1)  choosing  examples  and  nonexamples,  (2) 
remembering  the  critical  attributes  of  each  concept,  and  (3)  reviewing  concept 
definitions.  Student-assigned  strategies  are  to  be  utilized  by  the  student  without  system 
guidance.  The  student  has  complete  freedom  to  select  examples  and  nonexamples  in  any 
pattern. 

A  second  focus  was  a  comparison  of  tabular  and  hierarchical  presentations  of 
information.  In  an  hierarchical  display,  the  critical  attributes  required  to  make 
classification  decisions  are  represented  in  the  form  of  a  hierarchy.  Attributes  are 
represented  in  the  form  of  a  network,  which  shows  the  relationship  of  all  attribute  nodes 
and  branches.  In  a  tabular  display,  the  critical  attributes  required  to  make  classification 
decisions  are  represented  in  the  form  of  a  table.  Such  a  display  provides  the  information 
contained  in  each  node  but  detaches  the  nodes  from  the  hierarchy  and  represents  diem  in 
a  serial,  nonbranching  pattern.  The  classification  scheme  used  in  this  study  is  loosely 
based  on  the  imaginary  science  of  Xenograde  Systems  (Merrill,  1965).  As  implemented  in 
this  current  extention  of  the  original  Xenograde  "curriculum,"  the  scheme  groups 
imaginary  particle  systems  into  10  letters  of  the  alphabet  order  to  control  for  rote 
memory  effects. 

Subjects  included  two  experimental  groups  and  one  control  group,  all  of  whom  were 
required  to  leant  pseudo-concepts  instructionally  presented  via  an  Apple  II  micro¬ 
computer.  Subjects  in  the  first  experimental  group  were  presented  instruction  in  tabular 
farm  and  the  computer  system  assigned  learning  strategies.  The  second  experimental 
group  was  presented  with  instructional  information  in  hierarchical  form  and  the  group 
selected  their  own  learning  strategies.  The  control  group  was  presented  instruction  in 
tabular  form  and  the  group  employed  their  own  learning  strategies. 

Findings 

Results  indicated  that  the  direction  of  differences  between  group  means  was 
consistent  with  the  hypothesis  that  system-assigned  strategies  can  enhance  acquisition  of 
coordinate  concepts.  As  an  empirical  study,  the  investigation  produced  tentative 
evidence  that  it  is  possible  to  transform  a  conventional  instructional  strategy  based  on 
examples  and  nonexamples  into  an  equivalent  system-assigned  learning  strategy  in  which 
students  are  guided  in  the  selection  of  their  own  examples  and  nonexamples. 

The  study  encountered  two  major  methodological  problems,  both  of  which  were 
traced  to  the  use  of  continuation  school  students  as  subjects.  The  first  problem  was  that 
inadequate  control  of  individual  difference  variables  resulted  in  excessive  error  variance. 
Although  much  of  this  within-group  variance  was  clearly  attributable  to  differences  in 
prior  academic  achievement,  statistical  control  of  these  nuisance  effects  proved  infea¬ 
sible  because  of  the  complex  interaction  of  achievement  variables  with  the  experimental 
treatments.  It  was  suggested  that  this  problem  be  corrected  with  the  use  of  a  more 
homogeneous  subject  pool  or  with  the  use  of  statistical  controls  that  would  partial  out  the 
effect  of  the  interactions. 

The  second  problem  was  that  the  low  achievement  level  of  many  of  the  subjects  may 
have  violated  the  crucial  requirement  for  the  predicted  success  of  the  system-assigned 
strategies.  The  locus-of -processing  continuum  prescribed  embedded  treatment  for  low 
aptitude  students.  Therefore,  a  factor  that  may  have  contributed  to  the  lack  of  a 
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significant  treatment  effect  of  the  system-assigned  strategy  was  that  it  inappropriately 
intervened  for  some  of  the  students.  It  was  suggested  this  problem  be  rectified  with  the 
use  of  subjects  who  have  moderate  achievement  levels.  Prior  achievement  in  mathe¬ 
matics  appeared  to  be  an  excellent  predictor  of  the  ability  to  apply  the  system-assigned 
strategies  used  in  this  experiment. 

There  were  no  significant  differences  in  the  posttest  performance  of  groups  who  used 
the  hierarchical  and  tabular  displays.  Perhaps  the  most  surprising  result  of  this  study  was 
the  small  magnitude  of  the  treatment  effect.  It  is  unlikely  that  the  lack  of  a  significant 
effect  for  the  conceptual  hierarchy  was  due  to  error  variance  alone.  The  difference 
between  treatment  and  control  means  was  negligible.  A  more  likely  explanation  would  be 
that  the  hypothesized  positive  effect  of  the  hierarchy  as  an  expository  aid  was  confounded 
with  a  negative  effect  when  the  hierarchy  was  used  for  practice  in  instantiation 
definitions. 

Recommendations 


The  value  of  this  study  goes  beyond  specific  experimental  methodologies  and  results. 
It  serves  as  a  small-scale  illustration  of  some  of  the  practical  benefits  of  microcomputer- 
administered  research.  For  example,  the  classification  training  involved  procedures  that 
were  far  too  complex  to  implement  through  other  media  with  any  consistency,  yet 
subjects  required  a  minimum  of  supervision  to  operate  the  systems.  The  computer 
displays  themselves  (and  perhaps  their  billing  as  a  video  game)  appeared  to  maintain 
subject  attention  and  motivation  at  high  levels  throughout  the  experiment.  Data  indicate 
that  the  computer-administered  classification  test  was  an  extremely  reliable  instrument 
for  use  with  college  and  high- school  subjects  alike. 


CONCLUSIONS 

It  is  rare  that  any  one  research  study  can  significantly  influence  instructional 
practice.  The  four  studies  reported  herein  merely  set  a  foundation  for  future  research  in 
the  area  of  basic  skills  learning  by  low  aptitude  students.  It  is  clear  that,  if  significant 
resources  were  to  be  invested  in  the  modification  of  the  JOBS  curriculum,  the  justifica¬ 
tion  must  come  from  a  far  broader  investigation  resulting  in  less  ambiguous  results  than 
the  four  studies  presented  in  this  report. 

Since  the  JOBS  program  curricula  were  developed  with  the  use  of  the  most  current 
instructional  development  methodology,  it  will  probably  be  some  time  before  enough 
significant  research  is  conducted  and  synthesized  to  warrant  modification  of  the  curricula 
for  significant  impact  on  program  effectiveness.  When  and  if  such  research  becomes 
available,  the  question  of  curricula  modification  should  again  be  addressed. 


RECOMMENDATIONS 


It  is  recommended  that: 

1.  Curriculum  developers  do  not  alter  the  JOBS  curricula  at  the  present  time. 

2.  Instructional  research  personnel  use  the  studies  presented  in  this  report  to 
delineate  future  areas  of  instructional  research. 
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